Graphic layout designs play an essential role in visual communication. Yet handcrafting layout designs are skill-demanding, time-consuming, and non-scalable to batch production. Although generative models emerge to make design automation no longer utopian, it remains non-trivial to customize designs that comply with designers' multimodal desires, i.e., constrained by background images and driven by foreground contents. In this study, we propose \textit{LayoutDETR} that inherits the high quality and realism from generative modeling, in the meanwhile reformulating content-aware requirements as a detection problem: we learn to detect in a background image the reasonable locations, scales, and spatial relations for multimodal elements in a layout. Experiments validate that our solution yields new state-of-the-art performance for layout generation on public benchmarks and on our newly-curated ads banner dataset. For practical usage, we build our solution into a graphical system that facilitates user studies. We demonstrate that our designs attract more subjective preference than baselines by significant margins. Our code, models, dataset, graphical system, and demos are available at https://github.com/salesforce/LayoutDETR.

In this work, we propose MEDICO, a Multi-viEw Deep generative model for molecule generation, structural optimization, and the SARS-CoV-2 Inhibitor disCOvery. To the best of our knowledge, MEDICO is the first-of-this-kind graph generative model that can generate molecular graphs similar to the structure of targeted molecules, with a multi-view representation learning framework to sufficiently and adaptively learn comprehensive structural semantics from targeted molecular topology and geometry. We show that our MEDICO significantly outperforms the state-of-the-art methods in generating valid, unique, and novel molecules under benchmarking comparisons. In particular, we showcase the multi-view deep learning model enables us to generate not only the molecules structurally similar to the targeted molecules but also the molecules with desired chemical properties, demonstrating the strong capability of our model in exploring the chemical space deeply. Moreover, case study results on targeted molecule generation for the SARS-CoV-2 main protease (Mpro) show that by integrating molecule docking into our model as chemical priori, we successfully generate new small molecules with desired drug-like properties for the Mpro, potentially accelerating the de novo design of Covid-19 drugs. Further, we apply MEDICO to the structural optimization of three well-known Mpro inhibitors (N3, 11a, and GC376) and achieve ~88% improvement in their binding affinity to Mpro, demonstrating the application value of our model for the development of therapeutics for SARS-CoV-2 infection.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

Person re-identification plays a significant role in realistic scenarios due to its various applications in public security and video surveillance. Recently, leveraging the supervised or semi-unsupervised learning paradigms, which benefits from the large-scale datasets and strong computing performance, has achieved a competitive performance on a specific target domain. However, when Re-ID models are directly deployed in a new domain without target samples, they always suffer from considerable performance degradation and poor domain generalization. To address this challenge, we propose a Deep Multimodal Fusion network to elaborate rich semantic knowledge for assisting in representation learning during the pre-training. Importantly, a multimodal fusion strategy is introduced to translate the features of different modalities into the common space, which can significantly boost generalization capability of Re-ID model. As for the fine-tuning stage, a realistic dataset is adopted to fine-tune the pre-trained model for better distribution alignment with real-world data. Comprehensive experiments on benchmarks demonstrate that our method can significantly outperform previous domain generalization or meta-learning methods with a clear margin. Our source code will also be publicly available at https://github.com/JeremyXSC/DMF.

Generative Adversarial Networks (GANs) typically suffer from overfitting when limited training data is available. To facilitate GAN training, current methods propose to use data-specific augmentation techniques. Despite the effectiveness, it is difficult for these methods to scale to practical applications. In this work, we present ScoreMix, a novel and scalable data augmentation approach for various image synthesis tasks. We first produce augmented samples using the convex combinations of the real samples. Then, we optimize the augmented samples by minimizing the norms of the data scores, i.e., the gradients of the log-density functions. This procedure enforces the augmented samples close to the data manifold. To estimate the scores, we train a deep estimation network with multi-scale score matching. For different image synthesis tasks, we train the score estimation network using different data. We do not require the tuning of the hyperparameters or modifications to the network architecture. The ScoreMix method effectively increases the diversity of data and reduces the overfitting problem. Moreover, it can be easily incorporated into existing GAN models with minor modifications. Experimental results on numerous tasks demonstrate that GAN models equipped with the ScoreMix method achieve significant improvements.

我们提出了Patron，这是一种新方法，它使用基于及时的不确定性估计，用于在冷启动场景下进行预训练的语言模型进行微调的数据选择，即，没有初始标记的数据可用。在顾客中，我们设计（1）一种基于迅速的不确定性传播方法来估计数据点的重要性和（2）分区 - 然后 - 剥离（PTR）策略，以促进对注释的样品多样性。六个文本分类数据集的实验表明，赞助人的表现优于最强的冷启动数据选择基准，高达6.9％。此外，仅具有128个标签，顾客分别基于香草微调和及时的学习，获得了91.0％和92.1％的全面监督性能。我们的赞助人实施可在\ url {https://github.com/yueyu1030/patron}上获得。

最近，几种基于空间内存的方法已经验证了将中间框架及其面具作为内存有助于将视频中的目标对象细分目标对象。但是，它们主要集中于当前帧和内存框架之间的更好匹配，而无需明确关注内存质量。因此，较差的分割面罩的框架容易被记住，这导致了分割掩盖误差问题并进一步影响分割性能。此外，随着帧数的增长，内存框架的线性增加还限制了模型处理长视频的能力。为此，我们提出了一个质量感知的动态内存网络（QDMN）来评估每个帧的分割质量，从而使内存库可以选择性地存储准确的分段框架，以防止误差积累问题。然后，我们将细分质量与时间一致性相结合，以动态更新内存库以提高模型的实用性。我们的QDMN没有任何铃铛和哨子，在戴维斯和YouTube-Vos基准测试中都取得了新的最新性能。此外，广泛的实验表明，提议的质量评估模块（QAM）可以作为通用插件应用于基于内存的方法，并显着提高性能。我们的源代码可在https://github.com/workforai/qdmn上找到。

本文着重于由于看不见的分布变化而导致性能下降的图表上的分布概括。以前的图形域概括始终诉诸于不同源域之间的不变预测因子。但是，他们假设在培训期间提供了足够的源域，为现实应用带来了巨大挑战。相比之下，我们通过从源域中构造多个种群来提出一个新的图形域概括框架，称为DPS。具体而言，DPS旨在发现多个\ textbf {d} iverse和\ textbf {p}可redictable \ textbf {s}带有一组发电机的ubgraphs，即，子图是彼此不同的，但它们彼此不同，但所有这些都与相同的语义共享输入图。这些生成的源域被利用以学习跨域的\ textIt {Equi-Prestivical}图神经网络（GNN），这有望很好地概括到看不见的目标域。通常，DPS是模型不合时宜的，可以与各种GNN骨架合并。节点级别和图形基准测试的广泛实验表明，所提出的DPS为各种图形域概括任务实现了令人印象深刻的性能。

子图识别旨在发现对图表属性最具信息的图表的压缩子结构。可以通过使用相互信息估计器优化图形信息瓶颈（GIB）来配制它。然而，由于图数据的相互信息本质上难以估计，GIB遭到培训不稳定。本文介绍了一种噪声注入方法，用于压缩子图中的信息，这导致了一种新颖的变分图信息瓶颈（VGIB）框架。VGIB允许对其在温和假设下的物镜的易变分别近似。因此，VGIB享有更稳定和高效的培训过程 - 我们发现VGIB在练习中提高表演的速度快10倍。广泛的图形解释实验，图形神经网络的解释性，图表分类显示VGIB发现比现有方法更好的子图。